Learning from Data Sets with Missing Labels

This paper consider the task of learning discriminative classifiers of data when some class labels are missing from the data set(so-called “semi-supervised” learning), specifically when the labeled data are not drawn from the same distribution as the unlabeled data. This is an important issue in domains in which learning from only the labeled samples can result in a classifier that is not appropriate for the distribution of data to which it is to be applied. For example, lending institutions create models of who is likely to repay a loan from training sets consisting of people in their records who were given loans in the past; however, the institution only approved loan applications of those it judged likely to repay a loan. Learning from only approved loans yields an incorrect model because the training set is a biased sample of the general population of applicants, which is the population in which the model is to be used. Semi-supervised learning attempts to overcome this bias by including the unlabeled data in the learning process. This paper systematically explores the different types of bias that can arise in a semi-supervised setting, with examples of real-world situations. We use Bayesian networks to formalize each type of bias as a set of conditional independence relationships and for each case we present an overview of available learning algorithms is presented. These algorithms have been published in separate fields of research, including epidemiology, medical observational studies, econometrics, sociology, and credit scoring. 1 Semi-supervised learning Semi-supervised classifier learning is learning from a data set in which only some of the samples have class labels, which can occur for a variety of reasons. For example, this arises when building a model of whose loan applications to approve, or more precisely, building a model of applicants’ default/repayment behavior. (e.g. [4] [6] [7] [9]) When people apply for a loan, their application is either accepted or rejected, depending on the lender’s guess as to how likely the applicant is to repay the loan. Then the people whose applications were accepted either eventually repay the loan or default on the loan, which defines the two classes (good borrowers/bad borrowers). We would like to use some data-mining model to predict how likely a person is to repay a loan, so we can better decide whom to reject or accept. We learn the parameters to the model from a database created by a financial institution; however such databases only have repay/default behavior recorded for the people whose applications were accepted, since the rejected people never had a chance to repay or default on the loan. The accepts clearly constitute a biased sample of all the applicants, so using traditional supervised learning algorithms on only the labeled samples could lead to a biased model. We must use a learning algorithm that takes both the labeled and unlabeled samples into account. Using such semi-supervised